Bayesian learning in multi-layer perceptron neural network using Monte Carlo: mlp-mc-1

A Bayesian implementation of learning in neural networks using Monte Carlo sampling has been developed by Neal (1996). This computation intensive method has shown encouraging performance in (Neal 1996) and in a study using several datasets in (Rasmussen 1996). For a full description of the method the reader is referred to (Neal 1996). Here a brief description of the algorithm will be given, along with the heuristics employed. A feed forward multi-layer perceptron neural network with a single hidden layer of hyperbolic tangent units is used; the network is fully connected, including direct connections from the input to the output layer. The output units are linear. All units have biases. A network with a single output, I inputs and H hidden units implements the function